Functional verification is a method for ensuring the functional correctness of an integrated circuit, and occupies an important position in an integrated circuit design flow. A high-quality functional verification platform is a premise for ensuring the whole flow credible and reliable, and further ensuring the functional correctness of the integrated circuit. Mutation testing may evaluate the quality of the functional verification platform so that a verification engineer can quantitatively give evaluation criteria.
The basic steps of the current mutation testing are divided into three parts: 1) randomly selecting at least one mutant after generating mutants for multiple logic gates of a compiled integrated circuit under test. Here, the integrated circuit under test may be described with a Hardware Description Language (HDL), and may be compiled with tools such as NCVerilog, NCVHDL, Portals, etc. Here, the mutation refers to changing the function of a certain logic gate in the integrated circuit under test, for example, changing the function of an “AND” logic gate into an “OR/XOR/XNOR” logic gate. In a word, it is called a mutant as long as its function is different from the “AND.” Generating mutants for multiple logic gates of the compiled integrated circuit under test is the prior art, and it may be implemented using the existing methods for changing functions of logic gates and changing the number of inputs and outputs of the logic gates.
2) Performing a mutation testing simulation for the integrated circuit under test. The mutation testing simulation is very similar to an ordinary circuit simulation, and the only difference is enabling the mutation during the simulation. During each mutation testing simulation, only one mutant and one test case can be used. Here, the test case refers to, according to a test target, adding a series of inputs to the integrated circuit under test, and observing by a checker whether outputs of the integrated circuit under test conform to the expected states. Each group of input signals plus a corresponding checking policy is called one test case. For example, if it is to test the function of an “AND” logic gate having two input signals, it is necessary to add different values, i.e. <1, 1>; <1, 0>; <0, 1>; <0, 0> to the two input signals of the “AND” logic gate; each group of input values has a desired output check, for example, when the input is <1, 1>, the desired output is 1; when the input is <1, 0>, the desired output is 0, and so on. Here, each group of input values is the test case for the “AND” logic gate.
3) Giving quantitative evaluation criteria for the quality of the functional verification platform by synthesizing a series of logical function changes and their observation results. That is, steps 1 and 2 are executed recurrently to obtain the mutation testing results of a series of logic gates. Based on the obtained results, it is observed whether the functional verification platform can normally check a change in the logical function of the integrated circuit under test. For example, in the above “AND” gate case, if the input signal is <0, 1>, the output result of the “AND” gate should be 0, and the functional verification platform should also expect that the output result is 0. If the function of the “AND” gate is changed into “OR,” then the output is now changed into 1, and at this time it is necessary to observe whether the functional verification platform can normally report an error because the functional verification platform deems that the logical function is still the “AND.” If the functional verification platform does not report an error, it means that it does not normally detect the change of the logical function, and then there must be something wrong with the checker.
When performing the mutation testing, if it is to accurately observe whether the functional verification platform can detect changes of all logical functions, it is better to test all mutants against all test cases. Supposing that there exist 1 million mutants, using 50 test cases, 4 random seeds, and each simulation lasting 1 hour, then it needs to perform the mutation testing simulation for 200 million hours. When randomly selecting a mutant, if in a certain simulation, since the testing stimulus generated by the current test case cannot activate the mutant, the result of this simulation is consistent with that without mutation, and this simulation becomes an invalid simulation, and the mutant is called an Equivalent Mutant. The cause of the equivalent mutant is that the current test case does not add to the mutant with an input signal that may cause the function of the mutant to change. Again, taking the above “AND” logic gate as an example, if the “AND” is mutated to “OR,” while there are only two test cases, one is for the input signal <1, 1> and the other is for the input signal <0, 0>, then the outputs of the two input signals are the same for the “AND” and “OR”, and the functions do not change, and thus the mutant is not activated.
In order to improve efficiency for the quality evaluation of the functional verification platform, it is necessary to reduce equivalent mutants.